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ODAY THE U.S. MENTAL HEALTH SYSTEM IS IN A 
period of transition as policy makers, payors, and providers 
search for alternative models of short- and long-term care for 
people with acute and chronic mental illnesses. The past century has wit- 
nessed the rise and fall of several systems of care that held promise for 
varying periods of time and then fell out of favor (Morrissey and Gold- 
man 1984). The most ambitious effort to coordinate the delivery of 
mental health services on a national scale was the Community Mental 
Health Centers (CMHC) program initiated by the National Institute of 
Mental Health (NIMH) in the mid-1960s. The CMHC program man- 
dated comprehensive services, affiliation agreements between compo- 
nent organizations, catchmented responsibilities, and citizens boards. 
Such an arrangement would seem to be ideally suited to the multiple 
needs of persons with a chronic mental illness (CMI). Ironically, how- 
ever, during the formative years of the CMHC movement, these centers 
were never fully intended for nor utilized by these persons. 
At the outset, CMHCs were guided by a prevention and early treat- 
ment orientation, which called for services to be concentrated largely on 
heretofore unserved populations and those persons with acute, episodic, 
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and/or less severe psychiatric conditions. People with chronic or long- 
term conditions were served mainly by state and county mental hospt- 
tals. The mental hospital deinstitutionalization movement in the 1970s 
proceeded largely uncoordinated with the CMHCs (Chu and Trotter 
1974; Windle and Scully 1974; Morrissey 1982). In the 1980s, the grow- 
ing ranks of people who were homeless and mentally ill and the plight 
of persons with CMI in obtaining community-based services underscored 
the failings of the community mental health services system (Goldman 
and Morrissey 1985; Mechanic and Rochefort 1990). 

With the advent of the health and human services block grant to the 
states in 1980 at the outset of the Reagan administration, CMHC fund- 
ing shifted from the NIMH to state governments, which began to con- 
tract with CMHCs for services to persons with CMI. Yet, the CMHCs in 
many of our largest cities are often seen as impediments to, rather than 
vehicles for, alleviating these problems. The catchmenting of CMHC ser- 
vices along geographic lines, once advanced as a way of enhancing the 
availability and accessibility of mental health services, was now seen as a 
barrier to serving the CMI population. Catchmenting often encouraged 
buck passing or the denial of responsibility for problems seen as “out-of- 
area” of citywide in scope, such as those presented by persons who were 
both homeless and mentally ill. In addition, the plurality of providers in 
the larger mental health services field and their specialization by client, 
modality of treatment, and payor raise myriad continuity-of-care and co- 
ordination problems. 

Perhaps the most challenging aspect of serving persons with CMI, 
however, is that their needs are much broader than the traditional array 
of CMHC services can meet. In addition to medical-psychiatric care, 
these needs are for housing, income, job training, psychosocial rehabili- 
tation, and advocacy, among others (Bachrach 1980; Talbott 1980). 
Here, the problems are twofold. On the one hand. it is unlikely that ev- 
ery CMHC could or would develop this full array of staff and program 
expeftise given variations in their resources, locations, and commitments 
to public sector work. As a result specialized services vary widely in their 
availability from one catchment area to another in the same city. On the 
other hand, these social welfare services are organized as separate do- 
mains, each with its own special goals, source of categorical funding, eli- 
gibility requirements, geographic span, and individual mode of operation. 
Additional disjunctures are occasioned by gaps in the range of services 
available in many communities and the multiplicity of public, volun- 
tary, and private agencies that provide one or another of these services. 
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Rather than continuous and comprehensive care, the result is system 
fragmentation —a situation where responsibility is divided “among mul- 
tiple, separate individuals and agencies each with a categorical purpose, 
and the whole lacking a coherent policy, an integrated direction, and co- 
ordinated relationships” (Roemer, Kramer, and Frink 1975, 3). 

Over the years, there have been a number of efforts to develop ad- 
ministrative arrangements to overcome fragmentation in the way mental 
health services are financed and delivered. In the 1960s and 1970s, Cali- 
fornia led the nation in transferring budgetary control over both outpa- 
tient and inpatient care to county mental health departments. In the 
1970s, Wisconsin also evolved a system where both fiscal and program- 
matic responsibility for mental health services was fixed at the county 
level. During the same time period, Georgia and New York attempted 
to integrate state and community mental health services at the local level 
as a “balanced service system.” However, as of the early 1980s, all of 
these efforts were characterized as ‘utopias unrealized” (Talbott 1983) 
because of incomplete implementation, the perpetuation of state- 
operated services, and the lack of clear roles and administrative bound- 
aries between governmental and private sectors. 

The most recent and in many respects the most ambitious effort to 
create a truly comprehensive system of care for persons with CMI was 
launched by the Robert Wood Johnson Foundation (RWJF) on a pilot 
demonstration basis. The Program on Chronic Mental Illness (Aiken, 
Somers, and Shore 1986) was a unique public-private partnership be- 
tween the foundation and the U.S. Department of Housing and Urban 
Development (HUD). Nine cities from among the nation’s largest were 
selected on a competitive basis in 1986 to receive support in the form of 
grants, low-interest loans, and federal rent subsidies (Section 8 certifi- 
cates). The mandate was to develop communitywide systems of care, of- 
fering a broad range of health, mental health, social services, and housing 
options to help persons with CMI function more effectively in their every- 
day lives and avoid inappropriate institutionalization (Shore and Cohen 
1990). Each city was expected to create a local mental health authority, 
an entity that would assume central responsibility (fiscally, administra- 
tively, and clinically) for developing and coordinating these services in 
the public sector (Goldman, Morrissey, and Ridgely 1990). In addition, 
the local mental health authorities were expected to do the following: 


1. ensure continuity of care by designating a caregiver (case manager 
or team) responsible for coordinating services to meet client needs 
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2. create a flexible fimancimg system where funds can be moved to 
meet client needs 
. develop a range of Aousing options for CMI persons 
4. enhance the range of psychosocial and vocational rehabilitation 
programs to support clients in the community 
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The demonstration covered a five-year period from January 1987 
through December 1991, consisting of a two-year planning phase (1987- 
88) and a three-year operational phase (1989-91). 

What distinguished the demonstration from earlier efforts was its focus 
on the problems of chronic (i.e., serious and persistent) mental illness in 
large urban areas, its emphasis on coordinating social welfare and rehabili- 
tation services as well as mental health treatment, and its stress on the de- 
velopment of housing for this client population as a major responsibility 
for mental health system managers. The demonstration was essentially a 
service system intervention rather than a treatment intervention. No new 
or enhanced medical-psychiatric treatment was introduced in a consistent 
or controlled fashion, and none of the case management programs that 
were introduced rigorously followed the “assertive community-treatment” 
model developed in Madison, Wisconsin (Stein and Test 1980). the only 
model that has been shown to be effective in improving clinical out- 
comes and quality of life (Taube et al. 1990). Rather. as Rosenberger has 
noted, the RWJ/PCMI “was designed to address the fragmentation that 
the program’s developers considered the major impediment to the deliv- 
ery of adequate services for chronic mentally ill persons: fragmentation 
evidenced in the dissociation between funding authority and service re- 
sponsibility; in the gaps and discontinuities of care as patients move be- 
tween hospital, outpatient treatment, and rehabilitation programs; and 
in the inability of any one agency to ensure continuity of care across 
catchment areas” (Rosenberger 1990, 1171) . 

To overcome these problems, a two-level reorganization and expansion 
of the service system for persons with CMI was planned. At the client 
level, the goal was to achieve continuity of care across diverse agencies by 
developing case management programs so that clients with multuple 
needs could be assisted in obtaining services, entitlements, and housing. 
Virtually all of these case management programs relied upon a “broker” 
model (Robinson and Toff-Bergman 1992) whereby case managers, 
working alone or as a member of a team with a shared caseload. assist cli- 
ents in receiving services and entitlements. At the systems level, the local 
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mental health authority was to take responsibility for developing clini- 
cal, fiscal, and administrative arrangements whereby this array of com- 
munity agencies could be brought into a more stable and supportive role 
for persons with CMI in the local area. The local authority was to serve 
in the role of “systems case manager,” seeking to achieve coordination 
not only among agencies in the traditional mental health system (state 
and general hospitals, CMHCs, other clinics, day programs, and so 
forth) where coordination problems are legend (Dill and Rochefort 
1989; Greenley 1992), but also to draw in a much wider set of health 
and human service organizations (each organized loosely into separate 
sectors or domains) that heretofore had few connections or interdependen- 
cies with the traditional mental health service system. This broader set of 
agencies included social welfare, vocational and employment, housing, 
law enforcement and corrections, income maintenance, health care, and 
advocacy domains. 

The publicity surrounding the RWJ/PCMI led to the concept of a local 
mental health authority (LMHA) being touted as the systems development 
model for the 1990s. As a result, the LMHA concept is coming under a 
lot of scrutiny, and not all of the commentary to date has been favor- 
able. Rosenberger has criticized LMHAs as politically flawed because — 
like the CMHCs—they are an organizational solution imposed upon 
communities from above rather than a negotiated agreement between 
agencies and government units, which then voluntarily give up some of 
their authority and power so that systemwide needs can be accomplished 
(Rosenberger 1990). Dill and Rochefort (1989) make a related point in 
arguing that the most serious obstacle to services integration and consoli- 
dation of funding streams under a local mental health authority is “geo- 
political” —the operation of federal, state, county, and municipal public 
and private providers under many separate local jurisdictions. They note 
that rarely have the prerequisites been present for such integration to oc- 
cur. These include not only a special will and energy for system transfor- 
mation on the part of service workers, recipients, and citizens, but also 
mechanisms to organize and channel these feelings. As a result, Dill and 
Rochefort believe that the promise of local or substate mental health au- 
thorities still exceeds their performance. 

Greenley has questioned whether LMHAs will fulfill their promise be- 
cause neglected in their design are a number of organization and man- 
agement issues (Greenley 1992). Reflecting on the experience with LMHAs 
in Wisconsin (Stein and Ganser 1983), he points out that services are 
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highly variable across the state even though county-level LMHAs control 
and are responsible for service delivery. In his view, LMHAs will not suc- 
ceed unless managers and others in these systems go beyond administra- 
tive structures and “pay attention to issues of setting goals, monitoring 
care, giving feedback, and promoting desirable interorganizational cul- 
tures” (Greenley 1992, 382). 

Currently, there is a paucity of published data that can be used to as- 
sess the validity of these concerns. Most of these reservations rest on con- 
ceptual and theoretical grounds, rather than on quantitative assessments 
of the structure and performance of LMHAs. As a result, policy makers 
and program managers have little solid evidence on which to base deci- 
sions about adoption of the LMHA approach to services integration and 
system development or to evaluate the efficacy of such arrangements 
where they exist. 

For this reason, a lot of interest has been focused on the national eval- 
uation of the RWJ/PCMI demonstration (Goldman et al. 1990). As part 
of this evaluation, we have been conducting an assessment of the struc- 
ture and functioning of LMHAs in nine demonstration sites, the perfor- 
mance of the associated community support systems (CSSs), and the 
changes that occurred during the demonstration period. One of the 
novel features of this research is its incorporation of interorganizational 
and social network analysis techniques to measure service system struc- 
ture and change. This work builds upon theoretical and methodological 
advances during the past 15 years, which allow for the empirical study of 
service delivery systems as systems (Knoke and Rogers 1979; Burt 1982; 
Knoke and Kuklinski 1982; Lincoln and McBride 1985; Morrissey, Tausig, 
and Lindsey 1985; Doreian, Woodard, and Musa 1994). These methods 
allow us to characterize the overall structure and functioning of LMHAs 
and CSSs for persons with CMI at each site, and the patterns of resource 
flows and services coordination in the systems of care. Comparative anal- 
yses of these data enable us to document the interorganizational variabil- 
ity of CSSs across sites and, by linking to other data collected as part of 
the RWJF evaluation (Goldman et al. 1990), to determine whether ser- 
vice system improvements lead to positive client outcomes. 

This article focuses on a subset of these issues. We present findings 
from a key informant survey of LMHA performance and the extent of 
CSS change in the nine demonstration cities (Austin, Baltimore, Char- 
lotte, Cincinnati, Columbus, Denver, Honolulu, Philadelphia, and To- 
ledo) and one comparison site (Colorado Springs). For six of these cities 
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(five demonstration sites and the comparison community) we also 
present data from an interorganizational network study that assesses 
changes in the local CSSs vis-a-vis the goals of the RWJ/PCMI. These 
five demonstration sites (Baltimore, Cincinnati, Columbus, Denver, and 
Toledo) were selected early in the evaluation from the nine participating 
cities as the sites that showed the most promise for implementing the 
demonstration and as the ones that could best teach us general lessons 
about LMHA and CSS development for other cities throughout the 
country. For the same reasons, four of these cities were also selected as 
sites for the client outcome study sponsored by the National Evaluation 
of the RWJ/PCMI (see the article by Lehman et al. in this issue). Colo- 
rado Springs served as a comparison site for a separately funded client 
outcome evaluation in Denver (see the article by Shern et al. in this is- 
sue). A collaboration agreement between the National Evaluation of the 
RWJ]/PCMI and the Colorado study team was negotiated for sharing cli- 
ent outcome data. As part of this agreement, we included Colorado 
Springs as a comparison site for our service system change analyses. 

We draw upon both qualitative and quantitative data collected at two 
points in time to assess the performance of the LMHAs and their associ- 
ated CSSs in the two sets of cities. These analyses and results have both 
substantive and methodological value. Substantively, they provide a way 
of assessing the impact of the RWJ/PCMI and its uniformity across sites. 
Methodologically, we believe they illustrate a general strategy for analyz- 
ing changes in the structure and performance of CSSs and other such 
interorganizational service delivery arrangements. 


Methods 


Data Collection 


Multiple sources of qualitative and quantitative data were used to mea- 
sure the extent to which the LMHA and the CSS at each site changed in 
ways intended by the demonstration. Much of our contextual understanding 
of the sites came from information generated by the local RWJF programs 
and from site visits that we conducted over the course of the demonstra- 
tion (Goldman, Morrissey, and Ridgely 1990). The program-generated 
information consisted of the original application submitted to the foun- 
dation by each site; administrative reports and memoranda generated by 
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the LMHA, other agencies, and their consultants over the five-year pe- 
riod of the demonstration (and earlier, in many instances); quarterly 
reports submitted by each LMHA to the foundation identifying accom- 
plishments relative to program goals, obstacles encountered, and pro- 
posed solutions; and management information system data dealing with 
expenditures, client enrollment, and patterns of service utilization. 

We made at least three site visits to each city during the demonstra- 
tion, about once every 18 months. For the first visits, three of us worked 
as a team to interview the staff of the LMHAs and a cross-section of peo- 
ple from the community support system, including agency directors, 
board members, government officials, consumers, case managers, advo- 
cates, clinicians, program administrators, news media representatives, 
and family members. On subsequent occasions, we paired up to conduct 
the site visits on a rotation schedule. To gather qualitative information 
during these visits, we used rigorous procedures such as reinterviewing 
informants, debriefing site visitors in the field, verifying factual mate- 
rial, and sharing observations with external observers (Silverman, Ricci, 
and Gunter 1990). The purpose of these visits was to obtain an under- 
standing of the local systems of care through the eyes of stakeholders and 
to learn about the specific administrative and service system changes that 
the local RWJF program staff was trying to achieve during the demon- 
stration. Over time, we were also able to identify a number of similari- 
ties and differences between the sites in their history of mental health 
programming, the organizational ecology of the local mental health system, 
and the distinctive strategies that the authorities adopted to achieve the 
RWJ/PCMI goals (Goldman, Morrissey, and Ridgely 1990). These quali- 
tative observations helped us design two quantitative data-collection efforts 
at each site—a “key informant” survey (Morrissey et al. 1994; Ridgely, 
Goldman, and Morrissey 1992) and an interorganizational network survey 
of relationships among CSS agencies (Morrissey 1992). 

Key Informant Survey. The idea of expert opinion polling has a 
long tradition in the mental health needs assessment literature (Attkis- 
son et al. 1978). We adopted this strategy to obtain a set of performance 
ratings on how well the LMHAs were meeting the needs of CMI persons 
living in the demonstration sites and on the changes in the overall CSS 
that occurred during the operational phase of the demonstration. A “key 
informant survey” involving a mailed questionnaire was conducted at 
each site at two points in time (T1 and T2): mid-1989 (just prior to the 
operational phase of the demonstration) and again in mid-1991 (toward 
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the end of the demonstration). Although not a true pre-post compari- 
son, these data points allowed for an “early” versus a “late” performance 
rating for the LMHA and for the CSS at each site. 

The key informants included many of the people we interviewed dur- 
ing site visits plus a deeper sampling of stakeholders within sectors of the 
CSS. The sampling frame we used was designed to include persons with 
varying ro/es (board member, executive director, program director, clini- 
cal/direct service staff, other) and affi/iations (the mental health sector as 
well as other CSS domains such as social welfare, housing, rehabilitation, 
law enforcement/corrections, and so forth). About two-thirds of the re- 
spondents across sites consisted of persons occupying roles as agency pro- 
gram directors or clinical/direct service staff. The other respondents 
covered a diverse array of people such as government officials, advocates, 
consumers, family members, and local media representatives. With re- 
gard to respondents’ affiliations, about 60 percent of the sample across 
sites was drawn from the mental health sector (CMHCs, state and local 
hospitals, the LMHA, other mental health programs) and 40 percent 
were from other CSS sectors. The number and distribution of respon- 
dents by role and affiliation categories varied somewhat across sites de- 
pending upon response rates and the local constellation of services. The 
trend was for board members and elected officials to be underrepre- 
sented among respondents in each city. Otherwise, our analyses indicate 
that respondents were broadly representative of the target samples at 
both time periods. 

We applied the same selection criteria and sampling strategy for key 
informants in each city (regardless of the boundaries of the system as 
viewed by local participants) so that cross-city and cross-time compari- 
sons would be meaningful. We sought to create a panel design whereby 
respondents to the T1 and T2 surveys would be based on the same peo- 
ple or those occupying the same roles in the local system. Prior to the T2 
sutvey, we updated our mailing lists and replaced persons who were no 
longer available with substitutes who held similar positions. For several 
reasons (nonresponse, personnel turnover, new organizations being cre- 
ated between surveys, and other organizations becoming defunct), the 
percentage of T2 respondents who had responded at T1 varied from 46 
to 65 percent across cities. However, respondents were similar at both 
time periods based upon their roles (agency/program director, clini- 
cian/direct service provider, other) and affiliations (local mental health 
authority board/staff, other mental health agency, other CSS). Response 
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tates ranged from 59 to 73 percent across the 10 cities for the two surveys 
and, at both time periods, the returns were found to be representative of 
the survey universe. 

The key informant survey obtained respondent ratings on how well 
the LMHA and the local CSS were meeting the service needs of persons 
with CMI. Four scales were developed from these ratings: adequacy, 
quality, availability, and coordination. The adequacy and quality scales 
are based on ratings of each of the eleven CSS services— outreach, emer- 
gency, mental health treatment, psychosocial rehabilitation, case man- 
agement, basic needs, vocational, housing, medical, substance abuse, 
and supportive services (Stroul 1982). The availability and coordination 
scales, however, are based on more global judgments about the perfor- 
mance of the overall CSS. 

Adequacy is an 11-item index measuring each respondent’s answers, 
rated from (1) “all” to (5) “none,” to the question, “How many persons 
who need <each CSS service) actually receive them?” Qua/ity is an 
11-item index measuring answers to the question, “How would you rate 
the quality of care provided to CMI persons in (city name) with regard 
to each CSS service)?” response options ranged from (1) “very good” to 
(5) “very poor.” The respondent was asked to rate “quality” in terms of 
the extent to which each of the CSS services met current professional 
standards on interpersonal, technical, and physical location considerations. 

Both availability and coordination ate 10-item scales measuring te- 
sponses to the question, “How well does the current service system for 
CMI persons (in city) perform on these activities?”’; response categories 
ranged from (1) “very well” to (5) “very poorly.” The availability scale 
includes items such as “avoiding waiting lists,” “keeping red tape to a 
minimum,” “providing transportation,” and “making patients feel wel- 
come and at ease.” Items in the coordination scale measure the extent to 
which agencies work concertedly to collaborate in “creating opportunities 
for joint planning,” “ensuring timely access to client records,” “muni- 
mizing or eliminating conflicting rules and requirements,” and “devel- 
oping agreements to avoid needless duplication of effort.” 

Three additional four-item scales measuring the performance of the 
local mental health authorities in three key areas were constructed from 
responses to the question, “To what extent do you agree with the follow- 
ing statements about the (name of LMHA)?”; response options ranged 
from (1) “strongly agree” to (5) “strongly disagree.” Administrative au- 
thority includes “doing a good job in identifying the needs of CMI 
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persons,” “sorting out LMHA and provider agency powers and responsi- 
bilities,” “improving the ways local agencies work together,” and “the 
LMHA only adds another layer of bureaucracy” to the local system. Chiz- 
ical authority includes “ensuring 24-hour coverage for all CMI clients,” 
“expanding centralized case management to all CMI persons who need 
it,” “enhancing interagency discharge planning for clients and agen- 
cies,” and “having little impact on provision of basic housing, rehabilita- 
tion, and medical care for CMI persons.” Frsca/ authority includes 
“securing funds from multiple sources for CMI services,” “enforcing a 
more efficient use of current resources,” “encouraging system-wide fiscal 
planning,” and “assuring public providers about stable and predictable 
levels of funding for CMI services.” 

Simple additive scales were constructed for each measure. All compos- 
ite indexes were rescaled to the original five-point Likert scoring with in- 
dividual items reflected and transposed so that (1) indicates the lowest 
performance and (5) indicates the highest or most positive performance. 
The internal consistency coefficients for the four CSS scales ranged from 
.65 to .95 across the ten cities at each time period, and the coefficients 
for the three LMHA scales ranged from .52 to .89 (Morrissey et al. 1994). 

Interorganizational Network Survey. In addition to key informant 
ratings, we wanted to obtain measures of the extent to which interagency 
relations were becoming more coordinated and centralized under the in- 
fluence of the LMHA as the demonstration was carried out. We gathered 
data on the working relations between and among the LMHA and all of 
the agencies in the CSS in six of the ten cities. 

Data about the formal network of care at each site were gathered from 
a knowledgeable staff person (or “boundary spanner’) at each agency. 
Boundary-spanning roles (Aldrich and Herker 1977; Tushman and Scan- 
lan 1981) are those occupied by persons who serve as liaisons with other 
organizations or who are particularly knowledgeable about such relation- 
ships (e.g., agency, unit, or program directors). These respondents were 
identified from interviews conducted with representatives of the major 
agencies, recommendations made by project staff of the local RWJF 
demonstration, or direct inquiries with agency directors. 

A research team consisting of a coordinator/supervisor and two or 
three part-time interviewers was hired on site and trained to collect the 
network data in each city. An in-person interview was conducted with 
each respondent. The semistructured interview lasted approximately 45 
minutes and collected data on the respondent’s agency (e.g., mission, 
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number of employees, sources of funding, services provided to pefsons 
with CMI, size of client caseloads, the extent of involvement by the 
agency in the RWJF demonstration project). The interviewer then ori- 
ented the respondent to the matrix format and content of an interagency 
network questionnaire. The respondent was asked to complete the ques- 
tionnaire within two weeks, and the interviewers scheduled a time to 
pick up the completed questionnaire. In many instances, owing to vaca- 
tions, scheduling problems, and tardiness, it took longer to obtain com- 
pleted questionnaires. When 1989 respondents were not available for 
interviewing in 1991, their role successors within the same agency /sub- 
unit were substituted or another knowledgeable respondent was se- 
lected. In some instances, such as the city police department, responses 
were completed with input from several respondents. Ultimately, a re- 
sponse rate of 100 percent for each city was obtained at both time periods. 

We developed measures of LMHA and CSS performance from the 
network data to parallel the two measures from the key informant sur- 
vey. The LMHA measures are based on the agency respondents’ answers 
to three questions: (1) “How much influence does <the authority) have 
over mental health policies in this county?”; (2) “How helpful has <the 
authority) been in allowing your organization to attain its goals?”; and 
(3) “How well-coordinated is your agency with <the authority)?” The rat- 
ings were averaged across agency respondents in each city to get a sum- 
mary score for influence, helpfulness, and coordination that ranged from 
(1) “low” to (5) “high.” These three summary scores were again averaged 
to obtain a composite measure of LMHA performance for 1989 and 
1991. 

The measures of CSS performance are derived from an interorganiza- 
tional network analysis of the interagency relations among CSS agencies 
in each city. These analyses are based on data obtained in the network 
questionnaire relating to three sets of relations among CSS agencies: cli- 
ent referrals, shared information, and funds exchanges (Morrissey, Tau- 
sig, and Lindsey 1985). Client referrals are measured by responses to the 
question, “To what extent does your agency receive chent referrals from 
this other agency?” Information flows are measured by responses to the 
question, “To what extent does your agency receive :zformation for coot- 
dination, control, planning, or evaluation purposes from this other 
agency?” Funding exchanges are measured by the question, “To what ex- 
tent does your agency recezve funds (grants/contracts) from this agency?” 
Responses on these three questions are recorded on a five-point Likert- 
type scale ranging from (1) “not at all” to (5) ‘‘a lot.” 
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Each exchange relation (clients, funds, information) defines a separate 
network that comprises all local organizations serving persons with CMI. 
Client referral alone may not be a meaningful interagency relationship — 
it could even be a sign of agency dumping or uncoordinated care. By 
coupling referral data with information and funding relations, however, 
we end up focusing on multiplex or multibonded ties among the agen- 
cies in each network. For each site, therefore, six networks are analyzed 
consisting of the three exchange relations at two time points. The size of 
these networks 1s the same within each site, based on the number of CSS 
organizations surveyed. 

The responses to the network questions are arrayed into agency-by- 
agency matrices for clients, funds, and information exchanges, which are 
then stacked and submitted to a hierarchical clustering analysis (Schott 
1991; Calloway 1993). The clustering process groups agencies into “struc- 
turally equivalent positions” (Burt 1982) to the extent that within- 
position agencies have similar patterns of relations with all other CSS 
agencies in their client, fund, and information exchanges. A separate 
“blockmodel” is produced to represent the relations between and among 
these positions in each of the individual networks. The blockmodel rep- 
resents a structural simplification of the original agency-by-agency ma- 
trix; it preserves the underlying pattern of relations while reducing the 
number of “nodes” in the network. Although the number and (agency) 
composition of the positions remain the same in each blockmodel, the 
relations between positions may vary as a function of the specific roles 
the incumbent agencies play in each network. Blockmodels provide a 
convenient way of comparing networks both within and across cities. 

Based on information derived from these blockmodels, three indices 
are constructed to measure the structure of relations in each network: 


¢ Density: The extent of linkage or connectivity within the network 
among the structurally equivalent positions (Knoke and Kuklinski 
1982). This is measured as the proportion of the actual number of 
links compared with the total possible number of links in each net- 
work. The greater the number of links between positions, the 
higher the density. 

° Centralization: The extent to which a hierarchy of dependence ex- 
ists among the structural positions. It is interpreted as the extent to 
which one position, or set of positions, is the beneficiary or broker 
of all the relations in the system (Borgatti, Everett, and Freeman 
1992). It is measured as the number of times an agency lies on a 
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path between two other organizations in an exchange relation (for 
all dyadic pairs). The index varies from 0 (low centralization) to 1 
(high centralization). 

¢ Fragmentation: The extent to which positions within the system are 
isolated from each other rather than forming cohesive, “clique-like” 
groupings. Cliques are determined based on the proportion of ex- 
changes that occur within the position compared with those that oc- 
cur with other positions (Bolland and Wilson 1991). The index 
varies from 0 to 1 such that the higher the fragmentation index, the 
more “cliqueness” within the network. 


These indices correspond to three of the RWJ/PCMI goals and they 
are used here to capture the extent of system change during the demon- 
stration. The density index is a general measure of the extent of connec- 
tions or links among agencies in a network, something that the RWJ/ 
PCMI wanted to promote. The underlying assumptions are that the 
greater the density among CSS agencies, the more likely that persons 
with CMI will have access to services and that their multiple needs will 
be met. Another goal of the RWJF demonstration was for mental health 
authorities to centralize authority over the administrative, clinical, and 
financial aspects of the local CSSs. The cestrafzation index provides a 
way of assessing the extent to which this goal has been accomplished. A 
third major goal was to overcome the fragmentation of providers and ser- 
vices within and between the mental health sector and other sectors of 
the CSS. The fragmentation index provides a summary measure of the 
tendency for relations in a network to be concentrated within positions 
with no Setween-position linkages. The smaller the ratio of between- to 
within-position linkages, the greater the network fragmentation. 

These three structural indices were computed separately for three net- 
works of relations—client referrals, fund exchanges, and information 
flows. A composite measure based on the three indices was constructed 
by determining the extent to which each city had changed and whether 
this change was in the desired direction vis-a-vis the RWJ/PCMI goals. 
Consistent with the RWJ goals, the desired change from T1 to T2 was for 
density and centralization to zmcrease and for fragmentation to decrease 
in each network. To develop an appropriate composite measure, each of 
the three indices was first averaged across the client, funds, and informa- 
tion networks to obtain overall density, fragmentation, and centraliza- 
tion scores. Second, these three summary scores were again averaged to 
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obtain a composite measure for 1989 and for 1991. (Prior to this step, 
the fragmentation scores were adjusted to reflect positive gains.) The in- 
dividual scales vary from 0 to 1 so the composite can be thought of as a 
proportion — in this case, the proportion or extent to which each city ap- 
proximated the RWJF ideal at each time period. (To facilitate compari- 
sons, the composite scores are multiplied by 100 to convert decimals to 
whole numbers.) Movement toward the RWJ/PCMI ideal for each city is 
indicated by the difference between the 1989 and 1991 means for this 
composite measure. The more the structure of CSS agency relations had 
changed toward that ideal, the larger the resulting difference score. 


Analyses 


Analyses focus on the measures of LMHA and CSS performance based 
on key informant and interorganizational network data at two points in 
time. Key informant measures are based on ten sites, whereas the net- 
work measures are based on a subset of six sites. Difference-of-means 
tests are used to determine whether there are statistically significant dif- 
ferences between sites on each measure at T1 (1989) and whether T2-T1 
(1991-89) improvements occur for individual sites. As our respondent 
samples depart from a strict panel or repeated-measures design, we were 
concerned that observed differences between site measures and across 
time might be due to sampling artifacts rather than real effects. To assess 
this likelihood, we compared the ratings for respondents who partict- 
pated in the key informant survey in 1989 and 1991 with those who re- 
sponded only in 1989 and only in 1991. We found no significant 
differences between these subgroups across 60 comparisons (10 cities X 
2 measures X 2 time periods + 20 over-time differences). Consequently, 
we use all respondents for the analyses reported, thereby basing infer- 
ences on larger numbers and more stable estimates of differences across 
comparisons. The comparisons are based on the Kruskal-Wallis nonpara- 
metric means test, which provides a chi-square statistic and associated 
probability level. Given the sampling design followed in the study and 
because the data being compared in this article comprise scale scores 
with varying numbers of items and responses, we adopted a conservative 
approach to statistical testing and used nonparametric procedures. 

The city is the unit of analysis for the network index that reflects 
movement toward the RWJ/PCMI ideal. With only six composite scores 
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available for analysis, no statistical testing can be performed. Here, the 
percent change will be calculated to compare the movement of the CSS 
in each city toward the RWJ/PCMI deal and to compare the magnitude 
of change on this composite with the changes observed on other indices. 
For purposes of these comparisons, the mean values in table 4, which are 
based on a scale of 100 percent, have been converted into a 1-5 scale in 
figure 4 by dividing each mean by 20. 


Results 


Key Informant Ratings 


Table 1 presents the average performance of the LMHAs in the ten cities 
as rated by the key informants (also see figure 1). The performance rat- 
ings are averaged across the clinical, fiscal, and administrative subscales 
for 1989 and 1991. Inspection of the performance means for 1989 in- 
dicates that some significant differences existed between cities. Toledo 
(3.78) and Philadelphia (2.50) are outliers as they have the highest and 
the lowest scores, respectively. The other cities cluster into two groups— 
one with relatively 47gh performing LMHAs (Colorado Springs, Colum- 
bus, Charlotte, Cincinnati, and Austin) and the other with relatively /ow 
performing LMHAs (Honolulu, Baltimore, Denver). These are homoge- 
neous clusters in the sense that pairwise performance scores are signifi- 
cantly different Zetween clusters and outliers, but not wzthzn clusters, as 
indicated by the shaded bands in table 1. 

The comparatively high scores for the three Ohio sites are consistent 
with their head start on the demonstration. County ‘‘648” boards had 
been created in the mid-1960s by Ohio’s community mental health leg- 
islation and they had been channeling state mental health funding to 
local programs for 20 years. For two or three years prior to the demon- 
stration, senior staff of the Ohio Department of Mental Health had been 
working closely with the county mental health boards to get them to as- 
sume a much more active management role in the delivery of services to 
seriously mentally ill persons and to expand the array of community ser- 
vices for this population (Hogan 1992; Robinson 1991). These state- 
county activities intensified during the RWJ/PCMI application process, 
and they continued after the start of the demonstration. Four cities in 
Ohio responded to the program announcement from the foundation. 
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TABLE 1 
Performance of the LMHA at RWJ/PCMI Demonstration Cities 
as Viewed by Key Informants, 1989-91 


LMHA performance’ 


1989 1991 
Difference 
City N Mean (s.e.) Rank of means 
Toledo . , 57 3.78 °(.098) wie 
Colorado Springs 73 3.51 (.087) 5 —.03 
Columbus 81 3.44 (.083) 3 +.10 
Charlotte 52 3.41 (.103) 2 +.23 
Cincinnati 89 3.31 (.079) 8 —.20 
Austin «6. 


nv 


4 Comparisons: between sites (1989): chi-square = 137.5, 9 df, P< .0001; between sites 
(1991): chi-square = 131.9, 9 df, P< .0001; over time: * P< .05, ** P< .01. 
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FIG. 1. LMHA performance in 1989 versus 1991: key informant ratings. 0, T2 
(1991); @, T1 (1989). 
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Three of the applications were rated highly by the PCMI advisory board, 
but only two were initially selected for funding. The director and execu- 
tive staff of the Ohio Department of Mental Health (ODMH) saw this 
demonstration program as providing important leverage for their plans 
to transfer responsibility for state-operated mental hospitals to local com- 
munities (Robinson 1991). An agreement was reached with the foundation 
whereby the ODMH fully funded the third Ohio site with state resources. 

Denver, Baltimore, and Philadelphia were three sites that lagged be- 
hind the others in 1989. Two of these cities (Denver and Baltimore) had 

no true LMHA, and all three required considerable organizational change 
to acheive RWJ/PCMI goals for centralization of authority. As a result, 
much of the first two years of the RWJF planning grant at these two sites 
was devoted to sorting out governance issues and a variety of intergov- 
ernmental and interagency disputes surrounding the creation of a local 
public mental health authority. It is not surprising, then, that the per- 
formance of the LMHAs in these sites was rated lower than the Ohio sites 
in 1989. 

The LMHA performance rating for Colorado Springs is not for a RWJF- 
designated authority, but for the local CMHC, which is responsible for 
public health services in that city and surrounding county. This CMHC 
enjoys a national reputation for its management and service innovations. 
As table 1 illustrates, its LMHA performance rating was close to the 
demonstration site with the highest ratings (Toledo). 

The level of performance in the 1991 follow-up survey shows significant 
improvements as well as some slippage. The two largest mean differences — 
both increases — occurred for Philadelphia (+-0.75) and Baltumore (+0.55), 
and the largest decrease occurred for Denver (—.26). LMHAs in Hono- 
lulu (—.20) and Cincinnati (—.20) also had lower ratings in 1991, but 
the reduction was not statistically significant. Otherwise, the cities expe- 
rienced some small fluctuations in LMHA performance, but basically re- 
mained at their 1989 level. The overall pattern suggests that the sites 
lagging behind in 1989 caught up with the others by 1991, but those 
out front in 1989 didn’t experience significant improvements in LMHA 
performance over the two-year interval. 

These results are consistent with what we had learned from site visit- 
ing each city. Philadelphia had not made much progress during the first 
three years of the demonstration because of jurisdictional and resource 

disputes with the state mental health authority, turnover in key state 
and local mental health authority leadership positions, and a lot of pub- 
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lic controversy surrounding the closing of Philadelphia State Hospital. 
Then, with the closure of the hospital and the transfer of some $40 mil- 
lion to the LMHA, the authority and local “base service units” (formerly 
CMHCs) began to make major strides toward the RWJ/PCMI goals. Af- 
ter a delayed start and the hiring of a new director, Baltimore Mental 
Health Systems, Inc. (the local mental health authority) began to gain 
the confidence of CSS agencies and the state department of mental 
health. In Denver, a fiscal crisis precipitated by mid-year revenue fore- 
casts that could not sustain the increased units of service being delivered 
by the Mental Health Corporation of Denver, Inc. (the mental health 
authority) led to staff layoffs, plans for trimming the client rolls, and a 
reorganization of case management and clinic services. This resulted in a 
lot of negative media attention and public outcries about mismanagement. 

The Honolulu site experienced the most delays in implementing the 
demonstration, including a switch of the designated grantee organiza- 
tion and successive turnover in several of the key project leadership post- 
tions. Honolulu was unique among the RWJF sites in that all public 
mental health services in Hawaii (CMHCs as well as the one state mental 
hospital) were operated by the Division of Mental Health, a subunit of 
the State Health Department (Goldman, Morrissey, and Ridgely 1990). 
Most of the population in Hawaii is located in Honolulu, the state capi- 
tol on the island of Oahu. The role of the “local” mental health author- 
ity—initially assigned to the department of psychiatry at the University 
of Hawaii—was eventually assumed by the governor’s office and reas- 
signed to a subcabinet committee responsible for coordinating state 
health, welfare, and related departments. No permanent appointment 
to the top job in the Division of Mental Health was made for nearly 
three years following the start of the demonstration, in large part be- 
cause of recruitment difficulties and a low salary scale. The directorship 
of the RWJF demonstration project that was located within the division 
also changed hands several times during the demonstration. It is not sur- 
ptising, then, that the Denver and Honolulu key informant ratings de- 
clined in 1991. The ratings for the LMHA in Colorado Springs—the 
comparison site—remained relatively constant at both time periods, but 
at a high level. 

Table 2 and figure 2 present a summary of the key informant ratings 
for the performance of the overall community support system (CSS). The 
entries reflect the average ratings across the adequacy, quality, availabil- 
ity, and coordination scales for 1989 and 1991. The overall pattern ts 
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TABLE 2 
Performance of the CSS at RWJ/PCMI Demonstration Cities 
as Viewed by Key Informants, 1989-91 


CSS performance? 


1989 1991 
Se OT Difference 
City N Mean (s.e.) Rank N Mean (s.e.) Rank of means 
Todo Ss aan : oo 
Charlotte 53 3.02 .078 
Colorado Springs 63 3.00 .072 
Columbus 74 2.94 .066 
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Cincinnati 81 2.82 .063 85 2.87 .062 +.05 
Austin 53 2.82 .078 54 2.82 .078 0 
ee , a 7 ss 5 


- 


Mor ot BF oo? ‘ 61 as Bk ee es ee 
Philadelphia 75 2.44 .066 10 £79 +.31** 


* Comparisons: between sites (1989): chi-square = 89.3, 9 df, P < .0001; between sites 
(1991): chi-square = 77.9, 9 df, P< .0001; over ume: * P< .05, ** P< .01. 
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FIG. 2. CSS performance in 1989 versus 1991: key informant ratings. OQ. T? 
(1991); M@, Tl (1989). 
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nearly identical to the LMHA profile in table 1—the CSS performance 
ratings fall into the same four clusters, the CSSs in Philadelphia (+.31) 
and Baltimore (+.18) make over-time improvements (as does the CSS in 
Columbus at +.17), but the means for CSS performance in most of the 
other cities are not significantly different from their 1989 levels. The 
main difference is that the CSS ratings in each city tend to be /ower than 
the corresponding ratings for the LMHAs in table 1, with six of the nine 
demonstration cities still below the 3.0 midpoint of the scale at T2 
(1991). Clearly, the perception of key informants was that there was 
much more success in creating and empowering the LMHAs in this two- 
year period than in significantly improving the overall CSSs. 


Interorganizational Network Ratings 


Table 3 presents ratings of LMHA performance based on interorganiza- 
tional network data for the subset of six cities in 1989 and 1991 (also see 
figure 3). Like the key informant data, there are significant differences 
revealing three groups of cities, distinguished by the shaded areas of the 
table, performing at different levels in 1989. The LMHA ratings in 1989 
were variable — Cincinnati and Columbus had the highest ratings and 
Colorado Springs the lowest rating, with Toledo, Baltimore, and Denver 
at an intermediate level. Here, the LMHAs in three of the five demon- 
stration cities show sizable improvements by 1991— Toledo (+.61), Bal- 
timore (+.57), Denver (+.56)—while those in Cincinnati and Columbus 
remain the same. These results repeat the pattern found in the key infor- 
mant data, that is, the sites lagging behind in 1989 caught up with the 
others by 1991, but the early leaders did not improve their performance 
significantly. This finding is confirmed by the over-time analysis, which 
shows that the three lowest-rated demonstration sites (Toledo, Baltt- 
more, and Denver) in 1989 made statistically significant improvements 
over time. By 1991, the ratings for the five demonstration cities merge 
into a single homogeneous grouping such that the only statistically sig- 
nificant differences in LMHA performance are between Colorado Springs 
and these cities. 

However, this pattern masks some interesting differences between the 
two data sources. First, in contrast to the key informant data, the net- 
work ratings for the LMHA in Denver are much higher both in 1989 and 
1991, and they show a significant improvement over time. Second, the 
LMHA in Toledo shows a significant improvement on network, but not 
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TABLE 3 
LMHA Performance, 1989-91: Network Ratings 


LMHA performance? 


1989 1991 
TTT «SCOCO*D iffference Percent 
City N Mean (s.e.) Rank N Mean (s.e.) Rank of means change 


Toledo 36 3 ; 
Baltimore 61 3.29 (.138) 4 62 3.86 (.155) 2.5 +0.57 17.3* 
Denver 56 3.17 (.149) 5 53 3.73 (.162) 5 | 


Colorado 7 
‘g Springs 34,, 2.86 


@ Comparisons: between sites (1989): chi square = 23.59, 5 df, P =.02; between sites 
(1991): chi square = 20.58, 5 a@f, P= .01; over ume: * P< .05, ** P< .01. 


on key informant, data. And third, for all cities, the network ratings 

tend to be Azgher than the key informant ratings (especially in 1991). 
Table 4 presents the composite index for movement of the CSSs in the 

six cities toward the RWJ/PCMI goals of high density and centralization 
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FIG. 3. LMHA performance in 1989 versus 1991: network ratings. QO. T- 
(1991); @, T1 (1989). 
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TABLE 4 
CSS Change toward RWJF Goals, 
1989-91: Network Ratings 


CSS change 
1989 1991 

a Difference Percent 

City Mean Rank Mean Rank of means change 
Baltimore 47.78 1 57.78 l +10.00 20.9 
Columbus 47.56 2 46.78 4 —0.78 —1.6 
Cincinnati 45.00 3 52.44 3 + 7.44 16.5 
Denver 42.11 4 55.00 Z +12.89 30.6 
Toledo 37.67 ») 42.56 >) +4.89 12.9 
Colorado Springs 26.45 6 26.78 6 +0.33 1.3 


and low fragmentation (also see figure 4). At T1, the average of the 
composite scores for the five demonstration cities was 44.2 out of 100 
possible points. Toledo and Colorado Springs had the lowest composite 
scores in 1989, largely because the structure of the CSS networks in these 
two cities was much more decentralized than the other sites. CSSs in 
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FIG. 4. System change toward RWJF goals from 1989 to 1991: network ratings. 
O, T2 (1991); @, T1 (1989). 
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Denver (+12.89) and Baltimore (+10.0) experienced the most change, 
followed by Cincinnati (+7.44) and Toledo (+4.89). Colorado Springs, 
the comparison site, and Columbus remained essentially unchanged. 
When expressed as percent change scores, the CSS improvements in 
Denver (31 percent) and Baltimore (21 percent) are particularly impres- 
sive, and they equal or exceed the magnitude of any of the changes re- 
ported in other tables. It is interesting to note that neither city had a 
local mental health authority prior to the RWJF demonstration, whereas 
the other three demonstration cities had preexisting agencies that served 
as incipient authorities (Goldman, Morrissey, and Ridgely 1990). This 
suggests that the introduction of an authority may have its greatest struc- 
tural impact in settings where there has vo¢ been an overarching mental 
health planning or administrative unit responsible for services coordination. 
It can also be seen from table 4 that the five demonstration cities— 
both in 1989 and 1991—had Azgher composite scores than the compari- 
son community (Colorado Springs). This means that the structure of the 
client referral, information, and funding networks in these five demon- 
strations were much closer to the RWJ/PCMI system ideal of high 
centralization, low fragmentation, and high density. And, with the ex- 
ception of Columbus, the CSSs in the demonstration sites had at least a 
noticeable movement toward this ideal during the two-year interval. 
There are a number of other interesting trends that can be observed 
across the six cities that have both key informant and network data avail- 
able (see tables and figures 1-4). First, the most consistent finding ts 
that, regardless of time period (1989 or 1991) or sources of data (key in- 
formant or network), the ratings for the LMHAs are Azgher (and in many 
instances much higher) than the ratings for the CSSs in 23 of the 24 pos- 
sible comparisons (6 cities X 2 times X 2 measures). This 1s an important 
finding given the concerns that have been expressed in the literature 
about mental health authorities having trouble establishing themselves 
in their local service context and political environment (Dill and 
Rochefort 1989; Rosenberger 1990; Shore and Cohen 1990). Second, in 
9 of 12 possible cross-sectional comparisons involving CSS performance 
in these six cities, the key informant ratings are higher than the network 
ratings, whereas in 9 of 12 cross-sectional comparisons involving MHA 
performance in the same cities, the network ratings are higher than the 
key informant ratings. However, the T2-T1 change in MHA perfor- 
mance (5 of 6 comparisons) and in CSS performance (4 of 6 compati- 
sons) tended to be greater when estimated from the interorganizational 
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network data. Both data sources concur that the CSSs within the six sites 
were performing at a lower level than the MHAs. Stated differently, re- 
spondents were more likely to say that their MHAs were performing at a 
higher \evel than the overall service system for persons with CMI. 


Discussion 


The findings of this study provide clear and convincing evidence that 
most of the RWJ/PCMI demonstration sites were successful in creating 
local mental health authorities. The Honolulu site and, in some mea- 
sure, the Denver site were exceptions, but the LMHAs at the other seven 
sites were rated highly on clinical, fiscal, and administrative authority by 
the key informants, and were characterized as influential, helpful, and 
well coordinated with CSS agencies by agency directors and other “bound- 
ary spanner” personnel. In this sense, the findings challenge some of the 
more skeptical claims advanced in the literature about the flaws of LMHAs 
and their inability to gain wide support from stakeholders and service 
providers. 

Less positive evidence was found for the development of the CSSs 
during the evaluation. In most cities, the performance of the overall CSS 
was not rated as highly as the performance of the LMHA. This suggests 
that the creation of a LMHA as a discrete organization (or the assign- 
ment of the functions of a LMHA to an existing organization) ts much 
easier to accomplish than is the reorganization of the entire CSS along 
the lines envisioned in the RWJ/PCMI. Denver offers a contrasting case 
where the performance of the LMHA at T2 was downrated by key infor- 
mants for fiscal and management issues even though the network mea- 
sures indicate that the CSS experienced the most structural change of 
any city toward RWJ/PCMI goals. 

The absence of a true predemonstration baseline complicates the in- 
terpretation of the T2-T1 changes in LMHA and CSS performance, how- 
ever. Two interpretations of the findings are possible. One is that the T1 
(1989) survey occurred too late to capture the full impact of the demon- 
stration. That is, the big jump in LMHA/CSS performance may have oc- 
curred in the first year or two of the demonstration and we simply 
missed it. If so, the 1989-91 performance changes reported here would 
underestimate the true impact of the demonstration. The other interpre- 
tation is based on the notion of selection artifacts or preexisting charac- 
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teristics. That is, the high levels of LMHA/CSS performance observed 
actoss cities (especially for the Ohio sites) existed prior to the demonstra- 
tion, so that these performance levels cannot be logically attributed to 
the demonstration. In this view, the findings reported here overestimate 
the full impact of the demonstration. 

The absence of knowledge about potential upper bounds of LMHA 
and CSS change also complicates the interpretations and poses a variant 
of the “half-full/half-empty” argument. That is, if the expectation was 
that cities could traverse the full range of values on our measures within 
the demonstration period, then the amounts of change reported here 
would have to be considered modest in many instances. On the other 
hand, the possible existence of ceiling effects for planned changes in 
such large and complex city/county environments means that the RWJ/ 
PCMI sites actually achieved much of what was possible under the time 
and resource constraints of this demonstration. We suggested in earlier 
reports that five years was much too short a time period for the largest 
cities to accomplish the same results as the smaller ones (Goldman et al. 
1992; Goldman, Morrissey, and Ridgely 1990). The possibility of ceiling 
effects in the key informant findings is suggested by the tendency of the 
Ohio sites to remain high on LMHA/CSS performance at both time points, 
thereby showing very little change. Here again, most of the realizable 
gain may have occurred within the first two years of the demonstration, 
so the lack of large improvements in the two-year period examined in 
this study cannot be seen as evidence of ineffectiveness. This considera- 
tion also suggests that the /eve/ of performance attained rather than the 
T2-T1 change in performance may be a more realistic criterion for assess- 
ing the success of the demonstration. 

Another possibility is that the LMHA organizational form may only 
have dramatic effects in those settings where there is no prior history of 
local management of the mental health system/CSS. This observation 
certainly fits with the data profile for Baltimore and Denver, the two 
sites that had little or no prior experience with MHA-like arrangements. 
These two cities made the most dramatic gains in moving the structure 
of the local CSS toward RWJ/PCMI goals. Also, the LMHAs cannot be 
seen simply as a “top down” system change strategy. Local agencies em- 
braced the opportunities presented by the foundation, mobilized a com- 
munity-wide response to win a five-year grant and related benefits, and 
organized a core set of agencies to implement planned changes. The te- 
sult in most communities was the playing out of exogenous incentives 
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coupled with endogenous leadership and agency receptivity to develop a 
more systematic approach to caring for persons with CMI. 

Another interesting finding is that estimates of change based on net- 
work data do not yield identical results to the estimates based on key in- 
formant data. For LMHA performance, the T2-T1 differences for four of 
the cities—Columbus, Toledo, Denver, and Colorado Springs—are 
larger (1.e., more positive change) when estimated from the network rat- 
ings versus the key informant ratings (tables/figures 1 and 3). For CSS 
performance, the network ratings for each city tend to be much lower 
than the key informant ratings, but for four of the cities — Denver, Balti- 
more, Cincinnati, and Toledo—the amount of positive change is much 
larger when estimated from the network versus the key informant data 
(tables/figures 2 and 4). 

These disparities may be partially explained by differences both in re- 
spondents and in the underlying constructs being measured. The net- 
work data are obtained from a spokesperson representing each CSS 
agency, whereas the key informant data also include responses from a 
much broader set of stakeholders in each city. In many instances, agency 
directors may be in a better position to know about the day-to-day work- 
ing relations between the LMHA and CSS agencies and thereby have a 
better basis than the other key informants for rating LMHA performance 
higher. As to the underlying constructs, the network measure of CSS 
performance is based on the structure of interagency relations, and it re- 
flects movement toward the RWJF ideal, whereas the key informant ratings 
reflect perception about the overall adequacy, quality, and accessibility 
of the service system. The former is a structural measure based on the 
day-to-day working relations and division of labor among agencies, 
whereas the latter is more of a global impression of system performances. 
The perceptions of a diverse group of stakeholders may be affected by a 
host of factors, only some of which may be related to actual system 
change. These might include the regard in which administrators are 
held, the quality of clinical services, and events occurring in the service 
delivery environment that are unrelated to the RWJ/PCMI. As such, the 
key informant and network measures are actually tapping distinct aspects 
of service systems, and they may not yield identical results. 

One of the core assumptions of the RWJ/PCMI demonstration ts that 
centralized authority is the most effective way to organize services for 
CMI persons. This assumption was based on the idea that just as the uni- 
tary command structure associated with a well-functioning hospital can 
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integrate diverse services, disciplines, and levels of patient care, so too 
can it be applied to the need for integration of diverse services on a com- 
munitywide basis. However, the issue of whether centralization (“tight 
coupling”) or decentralization (“loose coupling’’) is the most effective 
way of organizing human service delivery systems is much debated 
among organizational theorists (Aldrich 1978). Most empirical service 
delivery systems are mixtures of the two organizational forms, and con- 
vincing research evidence for the superiority of one form over another is 
not readily available. Toledo and Colorado Springs were two of the sites 
in this study that were much more decentralized at the outset of the 
demonstration and did not change as much as other sites toward 
RWJ/PCMI ideals. When centralization is paired with other goals, like 
those articulated by the RWJ/PCMI of decreasing fragmentation and in- 
creasing density among agencies, trade-offs must be faced. Our quanti- 
tative network approach has sensitized us to the fact that density and 
centralization in service delivery systems cannot both be maximized at 
the same time; as density increases centralization decreases and vice 
versa. The same is true for fragmentation and centralization. These 
trade-offs were not recognized by the architects of the RWJ/PCMI, who 
saw these goals as compatible and simultaneously attainable. However, 
how to balance these competing directives in practice became a major 
challenge for the LMHAs in this demonstration as they would for any 
similar organizational entities seeking to manage the entire mental 
health and community support service system in large population areas. 
Clearly, program managers and researchers must give further thought to 
specifying realistic goals and appropriate performance criteria for assess- 
ing the effectiveness of LMHAs. 

Discussion of relations between system structure, system change, and 
system effectiveness brings us to a central issue in the RWJ/PCMI dem- 
onstration. One of the main goals of the national evaluation of the 
RWJ/PCMI was to assess whether service system changes actually lead to 
enhanced client outcomes such as quality of life and psychosocial func- 
tioning (see articles by Lehman et al. and Shern et al. in this issue). Im- 
plicit in the RWJ/PCMI is the assumption that system change toward 
demonstration goals will lead to improved client outcomes (Goldman, 
Morrissey, and Ridgely 1990). The logic model for the RWJ/PCMI actu- 
ally involves two steps that assume: (1) system change toward the 
RWJ/PCMI ideal will enhance continuity of care for persons with CMI, 
and (2) improvements in continuity of care will lead to enhanced client 
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outcomes. The findings that LMHA performance and change generally 
exceed CSS performance and change suggest that the intervening sys- 
tems changes were incomplete and that the impact on clients will be at- 
tenuated (see the articles by Lehman et al. and Shern et al. in this issue). 
Building multivariate models to test these hypotheses and alternative 
measures is now in process, and the results will be presented in a future 
paper. 

Several analyses trying to explicate the multiple impacts of the 
RWJ/PCMI are currently under way, so answers to many other questions 
of interest about local mental health authorities and service system 
change cannot be answered at this time. Nonetheless, it is clear from the 
data we have presented that most of the local mental health authorities 
created under the RWJ/PCMI attained high levels of performance 
within the five-year demonstration period. This evidence challenges the 
skepticism that has surfaced in several quarters. What is yet unclear is 
whether local mental health authorities will be any more effective or 
have any greater staying power as a way of promoting service-system im- 
provements for persons with CMI than did the systems of care champi- 
oned in earlier generations. 
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